Electronic Mail

• There are a number of jobs to do in getting email from one place to another
• It is helpful to discuss each job separately
  • Often, the different jobs are done by different programs
• The chart on p. 538 of the text diagrams the relationship between all of these pieces
• Mail User Agents (MUAs)
  • The program the user uses to read and compose messages
  • Popular examples include pine and mutt on Unix, Eudora and Outlook Express on PCs
  • Web-based mail programs like Hotmail are also MUAs
  • Users often get attached to a particular user agent, and want their favorite on your system. Unless you enjoy insanity, you're probably best off sticking with one or two that you "support" (keep current, answer questions about, etc.)
• Mail Transport Agents (MTAs)
  • MTAs do the work of getting messages from the sender to the correct host
  • Tasks include parsing email addresses, formatting email messages, and speaking to remote hosts
    • The conversation between hosts is either Simple Mail Transport Protocol (SMTP) or Extended SMTP (ESMTP)
    • Choices of MTAs for Unix include sendmail, smail, qumail, Postfix, zmailer, and a host of others
    • sendmail is the standard, comes with every flavor of Unix, and is the most widely used
• Delivery Agents
  • Once the mail message gets to the appropriate host, it must be delivered to the appropriate user's "inbox" (usually a file, but could be a database)
  • This task is handled by the delivery agent
  • Possiblities on Unix include /bin/mail (which is also the original user agent for Unix), procmail, mail.local, and smrsh
  • The delivery agent must understand how the mail is stored on the system and do whatever is appropriate to put a mail message into the right place, in the right format
  • The standard mail storage system is the directory /var/spool/mail (or sometimes /var/mail), with each user's mail stored in a file named the same as the user's login name. This file can contain multiple messages, in what's known as "mbox" format. Each message is identified by the characters "From " at the start of a line.
    • This system, unfortunately, doesn't scale very well. (Performance stinks with a huge number of users, or a huge number of messages in the inbox.) Getting better performance usually means a home-grown scheme, which also means customized versions of the user, delivery, and access agents.
• Access agents
  • In situations where users do not access their mail directly from the storage area, an access agent is needed.
  • The protocol usually used is either Internet Message Access Protocol (IMAP) or Post Office Protocol (POP)
    • The primary difference is that IMAP allows messages to be kept on the mail server and delivered or manipulated individually, while POP is designed for sending the entire contents of the inbox to the client computer
• Mail submission agents
  • In high-traffic situations, the job of handling messages from MUAs (including parsing mail addresses, qualifying hostnames, formatting headers, etc.) is enough that for the MTA to do this in addition to actually sending the mail proves to be too much.
  • RFC 2476 proposes separating out a Mail Submission Agent to handle receiving messages from users, leaving the MTA to handle transport of properly-formatted messages.

• History
  • Written by Eric Allman
  • Version 8 was a major rewrite in 1993
    • Still the standard
• Latest version is available from http://www.sendmail.org/
• Installation
  • A little different from most other packages
  • Uses a script called Build
    • You should not use make directly
  • Configuration of the build is done with C
  • Sendmail comes with pretty good installation instructions. Read them.
  • Build install installs the compiled code and config files
• running sendmail
  • Sendmail is usually run with a command like sendmail -bd -q15m
    • -bd runs sendmail as a daemon, listening for SMTP connections
      • on TCP port 25, by default
    • -q15m tells sendmail to retry sending queued mail every 15 minutes
      • sendmail also understands 'h' for hours, 'd' for days, 's' for seconds, and even 'w' for weeks
      • Values can be combined
        • -q1h30m would tell sendmail to run the queue every hour and a half
  • Other "modes" of operation are available and described in the text
• Configuration
  • sendmail's configuration is named sendmail.cf
  • It is notoriously cryptic
    • It was designed to be easy to parse
      • Written in the days when processing power was much more scarce than today
    • To combat this, most of the configuration has been moved to the use of macros using the C
    • Makes things easier for humans, but requires more processing power
    • The m4 macros get turned into a sendmail.cf, so that nothing has had to change in sendmail itself
• Sendmail comes with sample C
• The book gives a good explanation of most configuration situations

• A mail message has three distinct parts
  • The envelope
    • Defines who the message is from, who it is to, and where it should be returned if undeliverable
    • Not visible to users
    • Part of the SMTP dialog between MTAs
  • The headers
    • Various bits of information about the message
    • Formated as "Label: contents" (with the option of continuation lines)
    • Some headers are interesting ("From:", "Date:", "Subject:"), others much less so ("X-Mailer:", "Message-Id:", "Received:")
      • Though "Received:" can be important for debugging or for "email forensics"
    • Sender and recepient(s) may be listed in the header ("From:" and "To:", respectively), but it's what's in the envelope that really matters
      • That's how you can get spam addressed to something like "[email protected]"
      • Your address was in the envelope, and the "To:" header is just bogus junk
  • The body
    • The actual content of the message
    • Must be plain text
      • Binary data (attactments, for example) must be encoded
• Here are the mail headers from some spam I got. In class, we'll go through what the lines mean.

From [email protected]  Wed Nov 22 05:56:41 2000
Return-Path: <[email protected]>
Received: from localhost (localhost [127.0.0.1])
        by localhost.localdomain (8.9.3/8.9.3) with ESMTP id FAA00783
        for <geoff@localhost>; Wed, 22 Nov 2000 05:56:41 -0800
From: [email protected]
Received: from mail.moscow.com
        by localhost with POP3 (fetchmail-5.3.1)
        for geoff@localhost (single-drop); Wed, 22 Nov 2000 05:56:41 -0800 (PST)
Received: by whale.fsr.net (mbox geoff)
 (with Cubic Circle's cucipop (v1.31 1998/05/13) Wed Nov 22 05:55:55 2000)
X-From_: [email protected]  Thu Nov 16 16:22:33 2000
Received: from search10.sohu.com ([202.108.41.30])
        by whale.fsr.net (8.9.3/8.9.3) with ESMTP id QAA85681;
        Thu, 16 Nov 2000 16:22:27 -0800 (PST)
        (envelope-from [email protected])
Received: by search10.sohu.com id IAA0000013886; Fri, 17 Nov 2000 08:18:48 +0800 (CST)
Message-ID: <[email protected]>
To: <[email protected]>
Subject: BUSINESS OWNER/OPERATOR                         15628
Date: Thu, 16 Nov 2000 08:16:19 -0700
MIME-Version: 1.0
Content-Type: text/plain;
        charset="Windows-1252"
Content-Transfer-Encoding: 7bit
X-Priority: 3
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook Express 5.00.2615.200
X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2615.200

• In the context of email, "spam" refers to a message sent to hundreds or thousands of users. Most spam is what is called "unsolicited commercial email" (UCE). It's the internet equivalent of junk mail.
  • Spam is a resource drain in several ways
    • CPU time processing and delivering spam messages
    • Disk space used to store spam messages
    • "Help desk" time handling user complaints about spam
    • Administrator time trying to keep the system robust against the flood of spam
• It is nearly impossible to stem the flow of spam, but there are things that can be done to deal with it.
  • In a university setting, it can be more difficult to block spam, because any attempt to block certain kinds of email may be branded "censorship".
• Sendmail has added several features in recent versions to deal with the problem of spam.
  • Anti-relay features
    • Originally, sendmail would accept messages from anywhere and attempt to deliver them anywhere. This was part of the "friendly" nature of the internet
    • Spammers soon figured out that using a neutral system to send their messages had its advantages
      • It gave the spammer some anonymity
      • It used someone else's CPU cycles and bandwidth to send the mail
    • Starting with sendmail 8.9, sendmail by default allows no relaying
      • Hosts that you wish to allow to relay must be specified in sendmail's "access database" or in the file /etc/mail/relay-domains
  • Blacklisting
    • There are various services that provide lists of open relays or known spam sources
    • Sendmail can be set up to check incoming mail against these lists and refuse to accept mail from any site that's been "blacklisted"
  • Header checking
    • Sendmail can reject messages based on various header contents
    • Some examples (including obvious spam headers, and some email virus signatures) are listed on p. 603
• The text has an interesting discussion about interpreting the headers from spam messages, along with the warning that chasing spammers is a losing game